Introduction to Computer Vision: Plant Seedlings Classification¶

Problem Statement¶

Context¶

In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.

Objective¶

The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.

Data Dictionary¶

The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.

  • The dataset can be download from Olympus.
  • The data file names are:
    • images.npy
    • Label.csv
  • Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.

  • The goal of the project is to create a classifier capable of determining a plant's species from an image.

List of Species

  • Black-grass
  • Charlock
  • Cleavers
  • Common Chickweed
  • Common Wheat
  • Fat Hen
  • Loose Silky-bent
  • Maize
  • Scentless Mayweed
  • Shepherds Purse
  • Small-flowered Cranesbill
  • Sugar beet

Note: Please use GPU runtime to execute the code efficiently¶

Importing necessary libraries¶

In [ ]:
import os
import numpy as np                                                                               # Importing numpy for Matrix Operations
import pandas as pd                                                                              # Importing pandas to read CSV files
import matplotlib.pyplot as plt                                                                  # Importting matplotlib for Plotting and visualizing images
import math                                                                                      # Importing math module to perform mathematical operations
import cv2                                                                                       # Importing openCV for image processing
import seaborn as sns                                                                            # Importing seaborn to plot graphs


# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator                              # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential, Model                                            # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD                                                 # Importing the optimizers which can be used in our model
from sklearn import preprocessing                                                                # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split                                             # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score, roc_curve           # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
# Display images using OpenCV
from google.colab.patches import cv2_imshow                                                      # Importing cv2_imshow from google.patches to display images
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

Loading the dataset¶

In [ ]:
# Mount Google drive to access the dataset
# Run the below code if you using google colab
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
In [ ]:
# Load the image file of dataset
images = np.load('/content/drive/MyDrive/Python | AI | ML/Colab Notebooks/Computer Vision/Project/Plant_Images.npy')
# Load the labels file of dataset
labels = pd.read_csv('/content/drive/MyDrive/Python | AI | ML/Colab Notebooks/Computer Vision/Project/Plant_Labels.csv')

Data Overview¶

Understand the shape of the dataset¶

In [ ]:
print(images.shape)
print(labels.shape)
(4750, 128, 128, 3)
(4750, 1)
In [ ]:
cv2_imshow(images[1])
In [ ]:
plt.imshow(images[1])
Out[ ]:
<matplotlib.image.AxesImage at 0x7de5124eaa40>
In [ ]:
# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
  images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)

Exploratory Data Analysis¶

  • EDA is an important part of any project involving data.
  • It is important to investigate and understand the data better before building a model with it.
  • A few questions have been mentioned below which will help you understand the data better.
  • A thorough analysis of the data, in addition to the questions mentioned below, should be done.
  1. How are these different category plant images different from each other?
  2. Is the dataset provided an imbalance? (Check with using bar plots)
In [ ]:
def plot_images(images,labels):
  num_classes=10                                                                  # Number of Classes
  categories=np.unique(labels)
  keys=dict(labels['Label'])                                                      # Obtaing the unique classes from y_train
  rows = 3                                                                        # Defining number of rows=3
  cols = 4                                                                        # Defining number of columns=4
  fig = plt.figure(figsize=(10, 8))                                               # Defining the figure size to 10x8
  for i in range(cols):
      for j in range(rows):
          random_index = np.random.randint(0, len(labels))                        # Generating random indices from the data and plotting the images
          ax = fig.add_subplot(rows, cols, i * rows + j + 1)                      # Adding subplots with 3 rows and 4 columns
          ax.imshow(images[random_index, :])                                      # Plotting the image
          ax.set_title(keys[random_index])
  plt.show()
In [ ]:
plot_images(images,labels)
  • The plot above shows random images from the dataset.
In [ ]:
def plot_images_unique(images, labels):
    num_classes = 10  # Number of Classes
    categories = np.unique(labels)
    keys = labels['Label'].unique()  # Obtaining the unique classes from the 'Label' column

    rows = 3 # Defining number of rows=3
    cols = 4 # Defining number of columns=4

    fig, axes = plt.subplots(rows, cols, figsize=(10, 8))  # Defining the figure size to 10x8

    for i, category in enumerate(categories):
        # Find the indices of images with the current category
        category_indices = [index for index, label in enumerate(labels['Label']) if label == category]

        # Plot an image for the current category
        ax = axes[i // cols, i % cols]
        ax.imshow(images[category_indices[0], :])  # Display the first image of the category
        ax.set_title(str(category))

    # Hide empty subplots
    for i in range(len(categories), rows * cols):
        fig.delaxes(axes.flatten()[i])

    plt.show()
In [ ]:
plot_images_unique(images,labels)
  • The plot above shows the 12 unique classes from the dataset.
In [ ]:
print(labels['Label'].dtype)
object
In [ ]:
labels['Label'] = labels['Label'].astype(str)
In [ ]:
plt.figure(figsize=(10, 6))
sns.countplot(x='Label', data=labels, palette='Spectral')
plt.xticks(rotation='vertical')
plt.show()
  • As the barplot shows, the dataset is imbalanced in the distribution of the available images in certain classes, primarily Shepherds Purse, Common Wheat, Cleavers, Maize, and Black-Grass.

Data Preparation for Modeling¶

  • Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
  • You'll have to encode categorical features and scale the pixel values.
  • You will build a model using the train data and then check its performance

Split the dataset

In [ ]:
from sklearn.model_selection import train_test_split
X_temp, X_test, y_temp, y_test = train_test_split(np.array(images),labels , test_size=0.1, random_state=42,stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp,y_temp , test_size=0.1, random_state=42,stratify=y_temp)
In [ ]:
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
(3847, 128, 128, 3) (3847, 1)
(428, 128, 128, 3) (428, 1)
(475, 128, 128, 3) (475, 1)

Encode the target labels¶

In [ ]:
# Convert labels from names to one hot vectors.
# We have already used encoding methods like onehotencoder and labelencoder earlier so now we will be using a new encoding method called labelBinarizer.
# Labelbinarizer works similar to onehotencoder

from sklearn.preprocessing import LabelBinarizer
enc = LabelBinarizer()
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded=enc.transform(y_val)
y_test_encoded=enc.transform(y_test)

Data Normalization¶

In [ ]:
# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

Model Building¶

Model 1¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model1 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))

# Adding max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2), padding = 'same'))

model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model1.add(Dense(16, activation='relu'))
model1.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt=Adam()
# Compile model
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model1.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 64)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 64, 64, 32)        18464     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 544764 (2.08 MB)
Trainable params: 544764 (2.08 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
In [ ]:
history1 = model1.fit(
            X_train_normalized, y_train_encoded,
            epochs=30,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=32,
            verbose=2
)
Epoch 1/30
121/121 - 16s - loss: 2.4219 - accuracy: 0.1586 - val_loss: 2.3154 - val_accuracy: 0.2266 - 16s/epoch - 128ms/step
Epoch 2/30
121/121 - 2s - loss: 2.2147 - accuracy: 0.2826 - val_loss: 2.1083 - val_accuracy: 0.3271 - 2s/epoch - 18ms/step
Epoch 3/30
121/121 - 2s - loss: 2.0983 - accuracy: 0.3018 - val_loss: 2.0678 - val_accuracy: 0.3481 - 2s/epoch - 17ms/step
Epoch 4/30
121/121 - 2s - loss: 2.0260 - accuracy: 0.3140 - val_loss: 1.9203 - val_accuracy: 0.3668 - 2s/epoch - 17ms/step
Epoch 5/30
121/121 - 2s - loss: 2.0010 - accuracy: 0.3221 - val_loss: 1.8672 - val_accuracy: 0.3762 - 2s/epoch - 17ms/step
Epoch 6/30
121/121 - 2s - loss: 1.9615 - accuracy: 0.3280 - val_loss: 1.8477 - val_accuracy: 0.3762 - 2s/epoch - 17ms/step
Epoch 7/30
121/121 - 2s - loss: 1.9560 - accuracy: 0.3254 - val_loss: 1.8176 - val_accuracy: 0.3925 - 2s/epoch - 18ms/step
Epoch 8/30
121/121 - 2s - loss: 1.9050 - accuracy: 0.3275 - val_loss: 1.7660 - val_accuracy: 0.3832 - 2s/epoch - 19ms/step
Epoch 9/30
121/121 - 2s - loss: 1.8985 - accuracy: 0.3332 - val_loss: 1.7665 - val_accuracy: 0.3972 - 2s/epoch - 19ms/step
Epoch 10/30
121/121 - 2s - loss: 1.8558 - accuracy: 0.3460 - val_loss: 1.7090 - val_accuracy: 0.3972 - 2s/epoch - 17ms/step
Epoch 11/30
121/121 - 2s - loss: 1.8346 - accuracy: 0.3434 - val_loss: 1.6590 - val_accuracy: 0.4182 - 2s/epoch - 17ms/step
Epoch 12/30
121/121 - 3s - loss: 1.8187 - accuracy: 0.3494 - val_loss: 1.6596 - val_accuracy: 0.4065 - 3s/epoch - 23ms/step
Epoch 13/30
121/121 - 2s - loss: 1.8089 - accuracy: 0.3429 - val_loss: 1.6524 - val_accuracy: 0.4042 - 2s/epoch - 20ms/step
Epoch 14/30
121/121 - 2s - loss: 1.8048 - accuracy: 0.3392 - val_loss: 1.6473 - val_accuracy: 0.4136 - 2s/epoch - 18ms/step
Epoch 15/30
121/121 - 2s - loss: 1.7803 - accuracy: 0.3449 - val_loss: 1.6684 - val_accuracy: 0.3925 - 2s/epoch - 18ms/step
Epoch 16/30
121/121 - 2s - loss: 1.7825 - accuracy: 0.3535 - val_loss: 1.6707 - val_accuracy: 0.4065 - 2s/epoch - 19ms/step
Epoch 17/30
121/121 - 2s - loss: 1.7610 - accuracy: 0.3486 - val_loss: 1.5750 - val_accuracy: 0.4112 - 2s/epoch - 17ms/step
Epoch 18/30
121/121 - 2s - loss: 1.7596 - accuracy: 0.3470 - val_loss: 1.5803 - val_accuracy: 0.4276 - 2s/epoch - 17ms/step
Epoch 19/30
121/121 - 2s - loss: 1.7353 - accuracy: 0.3621 - val_loss: 1.5534 - val_accuracy: 0.4299 - 2s/epoch - 17ms/step
Epoch 20/30
121/121 - 2s - loss: 1.7240 - accuracy: 0.3647 - val_loss: 1.5911 - val_accuracy: 0.4136 - 2s/epoch - 17ms/step
Epoch 21/30
121/121 - 2s - loss: 1.7151 - accuracy: 0.3569 - val_loss: 1.5684 - val_accuracy: 0.4276 - 2s/epoch - 18ms/step
Epoch 22/30
121/121 - 2s - loss: 1.7104 - accuracy: 0.3644 - val_loss: 1.5922 - val_accuracy: 0.4159 - 2s/epoch - 18ms/step
Epoch 23/30
121/121 - 2s - loss: 1.6917 - accuracy: 0.3717 - val_loss: 1.5872 - val_accuracy: 0.4463 - 2s/epoch - 19ms/step
Epoch 24/30
121/121 - 2s - loss: 1.6921 - accuracy: 0.3759 - val_loss: 1.5355 - val_accuracy: 0.4416 - 2s/epoch - 18ms/step
Epoch 25/30
121/121 - 2s - loss: 1.6789 - accuracy: 0.3767 - val_loss: 1.5048 - val_accuracy: 0.4673 - 2s/epoch - 17ms/step
Epoch 26/30
121/121 - 2s - loss: 1.6359 - accuracy: 0.4053 - val_loss: 1.5006 - val_accuracy: 0.4977 - 2s/epoch - 17ms/step
Epoch 27/30
121/121 - 2s - loss: 1.6080 - accuracy: 0.4081 - val_loss: 1.5291 - val_accuracy: 0.4836 - 2s/epoch - 17ms/step
Epoch 28/30
121/121 - 2s - loss: 1.5977 - accuracy: 0.4214 - val_loss: 1.4772 - val_accuracy: 0.5164 - 2s/epoch - 17ms/step
Epoch 29/30
121/121 - 2s - loss: 1.5504 - accuracy: 0.4318 - val_loss: 1.4837 - val_accuracy: 0.5070 - 2s/epoch - 19ms/step
Epoch 30/30
121/121 - 2s - loss: 1.5718 - accuracy: 0.4276 - val_loss: 1.5596 - val_accuracy: 0.4883 - 2s/epoch - 19ms/step
In [ ]:
plt.plot(history1.history['accuracy'])
plt.plot(history1.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 1.6333 - accuracy: 0.5032 - 284ms/epoch - 19ms/step
In [ ]:
# Here we would get the output as probablities for each category
y_pred=model1.predict(X_test_normalized)
15/15 [==============================] - 0s 8ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 1¶

  • As an inital model, it has a fairly average performance at 42% training accuracy, 49% validation accuracy, and 50% test accuracy.
  • Based on the confusion matrix, the model is running into issues with false positives

Model 2¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model2 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
# Adding max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model2.add(Dense(16, activation='relu'))
model2.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model2.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt=Adam()
# Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model2.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 32)      18464     
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 32)        0         
 D)                                                              
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 32, 32, 32)        128       
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 563388 (2.15 MB)
Trainable params: 563324 (2.15 MB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [ ]:
history2 = model2.fit(
            X_train_normalized, y_train_encoded,
            epochs=30,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=32,
            verbose=2
)
Epoch 1/30
121/121 - 8s - loss: 2.2222 - accuracy: 0.2394 - val_loss: 2.4101 - val_accuracy: 0.1963 - 8s/epoch - 64ms/step
Epoch 2/30
121/121 - 4s - loss: 1.7732 - accuracy: 0.3660 - val_loss: 2.2091 - val_accuracy: 0.3364 - 4s/epoch - 33ms/step
Epoch 3/30
121/121 - 4s - loss: 1.4928 - accuracy: 0.4630 - val_loss: 2.2885 - val_accuracy: 0.2103 - 4s/epoch - 32ms/step
Epoch 4/30
121/121 - 4s - loss: 1.3418 - accuracy: 0.5152 - val_loss: 1.8699 - val_accuracy: 0.3224 - 4s/epoch - 32ms/step
Epoch 5/30
121/121 - 4s - loss: 1.2341 - accuracy: 0.5545 - val_loss: 1.1352 - val_accuracy: 0.6519 - 4s/epoch - 33ms/step
Epoch 6/30
121/121 - 4s - loss: 1.1078 - accuracy: 0.6075 - val_loss: 1.7773 - val_accuracy: 0.4696 - 4s/epoch - 33ms/step
Epoch 7/30
121/121 - 4s - loss: 1.0226 - accuracy: 0.6197 - val_loss: 1.9402 - val_accuracy: 0.4650 - 4s/epoch - 33ms/step
Epoch 8/30
121/121 - 4s - loss: 0.9205 - accuracy: 0.6681 - val_loss: 1.1610 - val_accuracy: 0.6425 - 4s/epoch - 32ms/step
Epoch 9/30
121/121 - 4s - loss: 0.8198 - accuracy: 0.7039 - val_loss: 1.8444 - val_accuracy: 0.5070 - 4s/epoch - 32ms/step
Epoch 10/30
121/121 - 4s - loss: 0.7662 - accuracy: 0.7138 - val_loss: 1.3635 - val_accuracy: 0.6168 - 4s/epoch - 35ms/step
Epoch 11/30
121/121 - 4s - loss: 0.6972 - accuracy: 0.7302 - val_loss: 1.3949 - val_accuracy: 0.6238 - 4s/epoch - 32ms/step
Epoch 12/30
121/121 - 4s - loss: 0.6870 - accuracy: 0.7450 - val_loss: 1.3392 - val_accuracy: 0.6565 - 4s/epoch - 32ms/step
Epoch 13/30
121/121 - 4s - loss: 0.6067 - accuracy: 0.7713 - val_loss: 1.5277 - val_accuracy: 0.5654 - 4s/epoch - 32ms/step
Epoch 14/30
121/121 - 4s - loss: 0.5640 - accuracy: 0.7939 - val_loss: 1.6589 - val_accuracy: 0.6028 - 4s/epoch - 34ms/step
Epoch 15/30
121/121 - 4s - loss: 0.5522 - accuracy: 0.7946 - val_loss: 1.7809 - val_accuracy: 0.5888 - 4s/epoch - 33ms/step
Epoch 16/30
121/121 - 4s - loss: 0.5487 - accuracy: 0.7941 - val_loss: 2.5506 - val_accuracy: 0.4930 - 4s/epoch - 32ms/step
Epoch 17/30
121/121 - 4s - loss: 0.5108 - accuracy: 0.8097 - val_loss: 2.0627 - val_accuracy: 0.5047 - 4s/epoch - 34ms/step
Epoch 18/30
121/121 - 4s - loss: 0.5019 - accuracy: 0.8149 - val_loss: 1.5702 - val_accuracy: 0.6495 - 4s/epoch - 36ms/step
Epoch 19/30
121/121 - 4s - loss: 0.4935 - accuracy: 0.8154 - val_loss: 2.0993 - val_accuracy: 0.6075 - 4s/epoch - 35ms/step
Epoch 20/30
121/121 - 4s - loss: 0.4837 - accuracy: 0.8183 - val_loss: 1.7613 - val_accuracy: 0.5888 - 4s/epoch - 32ms/step
Epoch 21/30
121/121 - 4s - loss: 0.4792 - accuracy: 0.8290 - val_loss: 1.9179 - val_accuracy: 0.5584 - 4s/epoch - 32ms/step
Epoch 22/30
121/121 - 4s - loss: 0.4511 - accuracy: 0.8284 - val_loss: 1.8315 - val_accuracy: 0.5561 - 4s/epoch - 33ms/step
Epoch 23/30
121/121 - 4s - loss: 0.4567 - accuracy: 0.8303 - val_loss: 1.7862 - val_accuracy: 0.5654 - 4s/epoch - 35ms/step
Epoch 24/30
121/121 - 4s - loss: 0.4266 - accuracy: 0.8373 - val_loss: 2.2484 - val_accuracy: 0.5981 - 4s/epoch - 32ms/step
Epoch 25/30
121/121 - 4s - loss: 0.4294 - accuracy: 0.8412 - val_loss: 1.8034 - val_accuracy: 0.6449 - 4s/epoch - 32ms/step
Epoch 26/30
121/121 - 4s - loss: 0.4149 - accuracy: 0.8448 - val_loss: 2.0269 - val_accuracy: 0.5981 - 4s/epoch - 33ms/step
Epoch 27/30
121/121 - 4s - loss: 0.4074 - accuracy: 0.8498 - val_loss: 1.8727 - val_accuracy: 0.6262 - 4s/epoch - 34ms/step
Epoch 28/30
121/121 - 4s - loss: 0.3652 - accuracy: 0.8622 - val_loss: 2.7845 - val_accuracy: 0.5070 - 4s/epoch - 33ms/step
Epoch 29/30
121/121 - 4s - loss: 0.3715 - accuracy: 0.8640 - val_loss: 2.0674 - val_accuracy: 0.6098 - 4s/epoch - 33ms/step
Epoch 30/30
121/121 - 4s - loss: 0.3674 - accuracy: 0.8653 - val_loss: 2.5580 - val_accuracy: 0.6005 - 4s/epoch - 33ms/step
In [ ]:
plt.plot(history2.history['accuracy'])
plt.plot(history2.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 2.7567 - accuracy: 0.6084 - 495ms/epoch - 33ms/step
In [ ]:
y_pred=model2.predict(X_test_normalized)
15/15 [==============================] - 0s 8ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 2¶

  • This model was revised to add 2 convolution layers before adding a maxpooling layer and also added batch normalization before the ANN layers to address the overfitting.
  • The changes in this model has improved the performance with the training accuracy at 87%, validation accuracy at 60%, and testing accuracy at 61%.

Model 3¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
train_datagen = ImageDataGenerator(
                              horizontal_flip=True,
                              vertical_flip=True,
                              zoom_range=0.75,
                              rotation_range=20,
                              fill_mode='nearest'
                              )
In [ ]:
# Intializing a sequential model
model3 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model3.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
model3.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
# Adding max pooling to reduce the size of output of first conv layer
model3.add(MaxPooling2D((2, 2), padding = 'same'))
model3.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model3.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model3.add(MaxPooling2D((2, 2), padding = 'same'))
model3.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model3.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model3.add(Dense(16, activation='relu'))
model3.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model3.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt=Adam()
# Compile model
model3.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model3.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 32)      18464     
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 32)        0         
 D)                                                              
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 32, 32, 32)        128       
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 563388 (2.15 MB)
Trainable params: 563324 (2.15 MB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [ ]:
# Epochs
epochs = 30
# Batch size
batch_size = 128

history3 = model3.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       seed=42,
                                       shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),
                    verbose=1)
Epoch 1/30
30/30 [==============================] - 24s 589ms/step - loss: 2.3500 - accuracy: 0.2019 - val_loss: 2.4653 - val_accuracy: 0.0841
Epoch 2/30
30/30 [==============================] - 19s 612ms/step - loss: 2.1290 - accuracy: 0.2681 - val_loss: 2.4346 - val_accuracy: 0.1776
Epoch 3/30
30/30 [==============================] - 18s 612ms/step - loss: 2.0140 - accuracy: 0.3084 - val_loss: 2.4100 - val_accuracy: 0.2009
Epoch 4/30
30/30 [==============================] - 18s 609ms/step - loss: 1.9259 - accuracy: 0.3410 - val_loss: 2.3832 - val_accuracy: 0.1729
Epoch 5/30
30/30 [==============================] - 18s 608ms/step - loss: 1.8656 - accuracy: 0.3614 - val_loss: 2.3892 - val_accuracy: 0.2827
Epoch 6/30
30/30 [==============================] - 18s 611ms/step - loss: 1.7888 - accuracy: 0.4012 - val_loss: 2.3216 - val_accuracy: 0.2407
Epoch 7/30
30/30 [==============================] - 18s 604ms/step - loss: 1.7790 - accuracy: 0.3985 - val_loss: 2.3193 - val_accuracy: 0.1799
Epoch 8/30
30/30 [==============================] - 19s 614ms/step - loss: 1.7119 - accuracy: 0.4230 - val_loss: 2.3610 - val_accuracy: 0.1262
Epoch 9/30
30/30 [==============================] - 22s 714ms/step - loss: 1.6444 - accuracy: 0.4396 - val_loss: 2.2978 - val_accuracy: 0.1425
Epoch 10/30
30/30 [==============================] - 18s 610ms/step - loss: 1.6207 - accuracy: 0.4439 - val_loss: 2.1899 - val_accuracy: 0.2336
Epoch 11/30
30/30 [==============================] - 18s 617ms/step - loss: 1.5668 - accuracy: 0.4585 - val_loss: 2.3125 - val_accuracy: 0.2313
Epoch 12/30
30/30 [==============================] - 18s 607ms/step - loss: 1.5623 - accuracy: 0.4665 - val_loss: 2.1595 - val_accuracy: 0.2407
Epoch 13/30
30/30 [==============================] - 30s 998ms/step - loss: 1.5141 - accuracy: 0.4732 - val_loss: 2.0366 - val_accuracy: 0.3294
Epoch 14/30
30/30 [==============================] - 20s 646ms/step - loss: 1.4639 - accuracy: 0.4977 - val_loss: 1.9530 - val_accuracy: 0.4322
Epoch 15/30
30/30 [==============================] - 18s 604ms/step - loss: 1.4814 - accuracy: 0.4945 - val_loss: 1.9067 - val_accuracy: 0.4019
Epoch 16/30
30/30 [==============================] - 18s 617ms/step - loss: 1.4457 - accuracy: 0.5069 - val_loss: 1.8728 - val_accuracy: 0.3505
Epoch 17/30
30/30 [==============================] - 18s 613ms/step - loss: 1.4740 - accuracy: 0.4870 - val_loss: 1.6365 - val_accuracy: 0.5187
Epoch 18/30
30/30 [==============================] - 19s 627ms/step - loss: 1.4379 - accuracy: 0.4993 - val_loss: 1.6905 - val_accuracy: 0.4159
Epoch 19/30
30/30 [==============================] - 22s 707ms/step - loss: 1.4318 - accuracy: 0.5069 - val_loss: 1.7418 - val_accuracy: 0.4019
Epoch 20/30
30/30 [==============================] - 18s 613ms/step - loss: 1.3875 - accuracy: 0.5233 - val_loss: 1.5510 - val_accuracy: 0.4136
Epoch 21/30
30/30 [==============================] - 19s 619ms/step - loss: 1.3157 - accuracy: 0.5450 - val_loss: 1.9297 - val_accuracy: 0.3341
Epoch 22/30
30/30 [==============================] - 19s 629ms/step - loss: 1.3154 - accuracy: 0.5485 - val_loss: 1.6098 - val_accuracy: 0.4603
Epoch 23/30
30/30 [==============================] - 20s 636ms/step - loss: 1.3179 - accuracy: 0.5531 - val_loss: 2.1655 - val_accuracy: 0.3178
Epoch 24/30
30/30 [==============================] - 19s 623ms/step - loss: 1.3448 - accuracy: 0.5445 - val_loss: 4.8545 - val_accuracy: 0.1238
Epoch 25/30
30/30 [==============================] - 18s 615ms/step - loss: 1.3991 - accuracy: 0.5305 - val_loss: 2.3973 - val_accuracy: 0.3528
Epoch 26/30
30/30 [==============================] - 19s 620ms/step - loss: 1.2736 - accuracy: 0.5579 - val_loss: 1.1037 - val_accuracy: 0.6168
Epoch 27/30
30/30 [==============================] - 19s 622ms/step - loss: 1.2642 - accuracy: 0.5668 - val_loss: 2.1385 - val_accuracy: 0.2734
Epoch 28/30
30/30 [==============================] - 19s 633ms/step - loss: 1.2577 - accuracy: 0.5700 - val_loss: 1.1402 - val_accuracy: 0.6285
Epoch 29/30
30/30 [==============================] - 19s 623ms/step - loss: 1.2437 - accuracy: 0.5840 - val_loss: 1.1927 - val_accuracy: 0.6168
Epoch 30/30
30/30 [==============================] - 19s 627ms/step - loss: 1.2182 - accuracy: 0.5859 - val_loss: 2.0179 - val_accuracy: 0.4533
In [ ]:
plt.plot(history3.history['accuracy'])
plt.plot(history3.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model3.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 2.0331 - accuracy: 0.4632 - 263ms/epoch - 18ms/step
In [ ]:
y_pred=model3.predict(X_test_normalized)
15/15 [==============================] - 0s 12ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 3¶

  • This model used the same structure as Model 2, but added data augmentation to train the model
  • False positives are present within this model structure

Model 4¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model4 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model4.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
model4.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
# Adding max pooling to reduce the size of output of first conv layer
model4.add(MaxPooling2D((2, 2), padding = 'same'))
model4.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model4.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model4.add(MaxPooling2D((2, 2), padding = 'same'))
model4.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model4.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model4.add(Dense(16, activation='relu'))
model4.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model4.add(Dense(12, activation='softmax'))

#using SDG optimizer
opt = SGD(learning_rate=0.01, momentum=0.9)
# Compile model
model4.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model4.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 32)      18464     
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 32)        0         
 D)                                                              
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 32, 32, 32)        128       
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 563388 (2.15 MB)
Trainable params: 563324 (2.15 MB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [ ]:
# Epochs
epochs = 30
# Batch size
batch_size = 128

history4 = model4.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       seed=42,
                                       shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),
                    verbose=1)
Epoch 1/30
30/30 [==============================] - 21s 643ms/step - loss: 2.4236 - accuracy: 0.1592 - val_loss: 2.7105 - val_accuracy: 0.1379
Epoch 2/30
30/30 [==============================] - 20s 666ms/step - loss: 2.2585 - accuracy: 0.2277 - val_loss: 2.9497 - val_accuracy: 0.1379
Epoch 3/30
30/30 [==============================] - 18s 606ms/step - loss: 2.1569 - accuracy: 0.2603 - val_loss: 3.4194 - val_accuracy: 0.1379
Epoch 4/30
30/30 [==============================] - 18s 614ms/step - loss: 2.1469 - accuracy: 0.2358 - val_loss: 4.3010 - val_accuracy: 0.1379
Epoch 5/30
30/30 [==============================] - 19s 622ms/step - loss: 2.0584 - accuracy: 0.2780 - val_loss: 2.3405 - val_accuracy: 0.2173
Epoch 6/30
30/30 [==============================] - 19s 631ms/step - loss: 1.9832 - accuracy: 0.2939 - val_loss: 3.1069 - val_accuracy: 0.1379
Epoch 7/30
30/30 [==============================] - 18s 614ms/step - loss: 1.9861 - accuracy: 0.2971 - val_loss: 2.4476 - val_accuracy: 0.1519
Epoch 8/30
30/30 [==============================] - 18s 600ms/step - loss: 1.9600 - accuracy: 0.3087 - val_loss: 2.4973 - val_accuracy: 0.2150
Epoch 9/30
30/30 [==============================] - 18s 615ms/step - loss: 1.9309 - accuracy: 0.3065 - val_loss: 9.0049 - val_accuracy: 0.1379
Epoch 10/30
30/30 [==============================] - 18s 612ms/step - loss: 1.9303 - accuracy: 0.3151 - val_loss: 4.5997 - val_accuracy: 0.0607
Epoch 11/30
30/30 [==============================] - 18s 607ms/step - loss: 1.9094 - accuracy: 0.3200 - val_loss: 4.7515 - val_accuracy: 0.1542
Epoch 12/30
30/30 [==============================] - 19s 605ms/step - loss: 1.8973 - accuracy: 0.3307 - val_loss: 2.5933 - val_accuracy: 0.2640
Epoch 13/30
30/30 [==============================] - 19s 618ms/step - loss: 1.8733 - accuracy: 0.3358 - val_loss: 11.4476 - val_accuracy: 0.0514
Epoch 14/30
30/30 [==============================] - 21s 699ms/step - loss: 1.8551 - accuracy: 0.3463 - val_loss: 7.9070 - val_accuracy: 0.0607
Epoch 15/30
30/30 [==============================] - 19s 617ms/step - loss: 1.8658 - accuracy: 0.3482 - val_loss: 2.5382 - val_accuracy: 0.3224
Epoch 16/30
30/30 [==============================] - 18s 603ms/step - loss: 1.9032 - accuracy: 0.3345 - val_loss: 6.5856 - val_accuracy: 0.1472
Epoch 17/30
30/30 [==============================] - 18s 600ms/step - loss: 1.8351 - accuracy: 0.3595 - val_loss: 20.6634 - val_accuracy: 0.1425
Epoch 18/30
30/30 [==============================] - 18s 605ms/step - loss: 1.8218 - accuracy: 0.3625 - val_loss: 4.9153 - val_accuracy: 0.1893
Epoch 19/30
30/30 [==============================] - 18s 601ms/step - loss: 1.7856 - accuracy: 0.3845 - val_loss: 15.0511 - val_accuracy: 0.1262
Epoch 20/30
30/30 [==============================] - 18s 599ms/step - loss: 1.7389 - accuracy: 0.4028 - val_loss: 7.3397 - val_accuracy: 0.1238
Epoch 21/30
30/30 [==============================] - 19s 624ms/step - loss: 1.7341 - accuracy: 0.4023 - val_loss: 8.3138 - val_accuracy: 0.1168
Epoch 22/30
30/30 [==============================] - 19s 627ms/step - loss: 1.7279 - accuracy: 0.4106 - val_loss: 3.9766 - val_accuracy: 0.1846
Epoch 23/30
30/30 [==============================] - 18s 603ms/step - loss: 1.6976 - accuracy: 0.4157 - val_loss: 18.9211 - val_accuracy: 0.1168
Epoch 24/30
30/30 [==============================] - 18s 614ms/step - loss: 1.7635 - accuracy: 0.4071 - val_loss: 16.1108 - val_accuracy: 0.1028
Epoch 25/30
30/30 [==============================] - 18s 603ms/step - loss: 2.1114 - accuracy: 0.2616 - val_loss: 31.7079 - val_accuracy: 0.0537
Epoch 26/30
30/30 [==============================] - 19s 633ms/step - loss: 1.7839 - accuracy: 0.3673 - val_loss: 15.2381 - val_accuracy: 0.0818
Epoch 27/30
30/30 [==============================] - 20s 664ms/step - loss: 1.6943 - accuracy: 0.3958 - val_loss: 26.4368 - val_accuracy: 0.1075
Epoch 28/30
30/30 [==============================] - 19s 616ms/step - loss: 1.6917 - accuracy: 0.4119 - val_loss: 13.3201 - val_accuracy: 0.1402
Epoch 29/30
30/30 [==============================] - 19s 607ms/step - loss: 1.7584 - accuracy: 0.4006 - val_loss: 19.2734 - val_accuracy: 0.0888
Epoch 30/30
30/30 [==============================] - 19s 619ms/step - loss: 1.6518 - accuracy: 0.4195 - val_loss: 5.6886 - val_accuracy: 0.2383
In [ ]:
plt.plot(history4.history['accuracy'])
plt.plot(history4.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model4.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 5.7588 - accuracy: 0.2379 - 212ms/epoch - 14ms/step
In [ ]:
y_pred=model4.predict(X_test_normalized)
15/15 [==============================] - 0s 13ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 4¶

  • This model used the SDG optimizer along with the data augmentation and it increased the false positives and also performed poorly on the accuracy scores

Model 5¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model5 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model5.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model5.add(MaxPooling2D((2, 2), padding = 'same'))
model5.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model5.add(MaxPooling2D((2, 2), padding = 'same'))
model5.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model5.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model5.add(Dense(16, activation='relu'))
model5.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model5.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt = Adam()
# Compile model
model5.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model5.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 64)        0         
 D)                                                              
                                                                 
 conv2d_1 (Conv2D)           (None, 64, 64, 32)        18464     
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 32, 32, 32)        128       
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 544892 (2.08 MB)
Trainable params: 544828 (2.08 MB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [ ]:
# Epochs
epochs = 30
# Batch size
batch_size = 128

history5 = model5.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       seed=42,
                                       shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),
                    verbose=1)
Epoch 1/30
30/30 [==============================] - 20s 571ms/step - loss: 2.3523 - accuracy: 0.1788 - val_loss: 2.4396 - val_accuracy: 0.1869
Epoch 2/30
30/30 [==============================] - 18s 587ms/step - loss: 2.0984 - accuracy: 0.2592 - val_loss: 2.3821 - val_accuracy: 0.2617
Epoch 3/30
30/30 [==============================] - 18s 579ms/step - loss: 2.0206 - accuracy: 0.2955 - val_loss: 2.3669 - val_accuracy: 0.1612
Epoch 4/30
30/30 [==============================] - 18s 590ms/step - loss: 1.9505 - accuracy: 0.3275 - val_loss: 2.3517 - val_accuracy: 0.3458
Epoch 5/30
30/30 [==============================] - 18s 587ms/step - loss: 1.8589 - accuracy: 0.3469 - val_loss: 2.2980 - val_accuracy: 0.2664
Epoch 6/30
30/30 [==============================] - 18s 591ms/step - loss: 1.8399 - accuracy: 0.3617 - val_loss: 2.2984 - val_accuracy: 0.3505
Epoch 7/30
30/30 [==============================] - 18s 588ms/step - loss: 1.8230 - accuracy: 0.3670 - val_loss: 2.3469 - val_accuracy: 0.2243
Epoch 8/30
30/30 [==============================] - 17s 581ms/step - loss: 1.7633 - accuracy: 0.3786 - val_loss: 2.2897 - val_accuracy: 0.3879
Epoch 9/30
30/30 [==============================] - 21s 709ms/step - loss: 1.7460 - accuracy: 0.3947 - val_loss: 2.2246 - val_accuracy: 0.2827
Epoch 10/30
30/30 [==============================] - 18s 591ms/step - loss: 1.7286 - accuracy: 0.3928 - val_loss: 2.2293 - val_accuracy: 0.3715
Epoch 11/30
30/30 [==============================] - 18s 592ms/step - loss: 1.7333 - accuracy: 0.4025 - val_loss: 2.2314 - val_accuracy: 0.3715
Epoch 12/30
30/30 [==============================] - 18s 600ms/step - loss: 1.6694 - accuracy: 0.4122 - val_loss: 2.1726 - val_accuracy: 0.4579
Epoch 13/30
30/30 [==============================] - 18s 587ms/step - loss: 1.6293 - accuracy: 0.4243 - val_loss: 1.8488 - val_accuracy: 0.3972
Epoch 14/30
30/30 [==============================] - 18s 586ms/step - loss: 1.6726 - accuracy: 0.4138 - val_loss: 1.9667 - val_accuracy: 0.4813
Epoch 15/30
30/30 [==============================] - 18s 589ms/step - loss: 1.6103 - accuracy: 0.4316 - val_loss: 2.2121 - val_accuracy: 0.2430
Epoch 16/30
30/30 [==============================] - 18s 591ms/step - loss: 1.6505 - accuracy: 0.4297 - val_loss: 1.8490 - val_accuracy: 0.4486
Epoch 17/30
30/30 [==============================] - 18s 595ms/step - loss: 1.6338 - accuracy: 0.4343 - val_loss: 1.7635 - val_accuracy: 0.4930
Epoch 18/30
30/30 [==============================] - 18s 586ms/step - loss: 1.5991 - accuracy: 0.4367 - val_loss: 1.5619 - val_accuracy: 0.5047
Epoch 19/30
30/30 [==============================] - 18s 591ms/step - loss: 1.5965 - accuracy: 0.4431 - val_loss: 1.8394 - val_accuracy: 0.5117
Epoch 20/30
30/30 [==============================] - 18s 593ms/step - loss: 1.5267 - accuracy: 0.4668 - val_loss: 1.4332 - val_accuracy: 0.4977
Epoch 21/30
30/30 [==============================] - 21s 707ms/step - loss: 1.4737 - accuracy: 0.4781 - val_loss: 1.5091 - val_accuracy: 0.4930
Epoch 22/30
30/30 [==============================] - 18s 595ms/step - loss: 1.5239 - accuracy: 0.4727 - val_loss: 1.4079 - val_accuracy: 0.5818
Epoch 23/30
30/30 [==============================] - 23s 763ms/step - loss: 1.5225 - accuracy: 0.4784 - val_loss: 1.3036 - val_accuracy: 0.5491
Epoch 24/30
30/30 [==============================] - 18s 583ms/step - loss: 1.4657 - accuracy: 0.4886 - val_loss: 1.5022 - val_accuracy: 0.5070
Epoch 25/30
30/30 [==============================] - 18s 584ms/step - loss: 1.4812 - accuracy: 0.4775 - val_loss: 1.2691 - val_accuracy: 0.5794
Epoch 26/30
30/30 [==============================] - 18s 594ms/step - loss: 1.4138 - accuracy: 0.5004 - val_loss: 1.1128 - val_accuracy: 0.6519
Epoch 27/30
30/30 [==============================] - 18s 598ms/step - loss: 1.4621 - accuracy: 0.4824 - val_loss: 1.1321 - val_accuracy: 0.6098
Epoch 28/30
30/30 [==============================] - 19s 606ms/step - loss: 1.4314 - accuracy: 0.5044 - val_loss: 1.1114 - val_accuracy: 0.6729
Epoch 29/30
30/30 [==============================] - 18s 586ms/step - loss: 1.4020 - accuracy: 0.5020 - val_loss: 1.4814 - val_accuracy: 0.5304
Epoch 30/30
30/30 [==============================] - 18s 597ms/step - loss: 1.4148 - accuracy: 0.5052 - val_loss: 1.5126 - val_accuracy: 0.4650
In [ ]:
plt.plot(history5.history['accuracy'])
plt.plot(history5.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model5.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 1.6379 - accuracy: 0.4400 - 134ms/epoch - 9ms/step
In [ ]:
y_pred=model5.predict(X_test_normalized)
15/15 [==============================] - 0s 7ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 5¶

  • This model used the network for model 1 with the addition of batch normalization and reverted back to the Adam optimizer

Model 6¶

In [ ]:
from keras.applications.vgg16 import VGG16

vgg_model1 = VGG16(weights='imagenet', include_top = False, input_shape = (128,128,3))
vgg_model1.summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58889256/58889256 [==============================] - 2s 0us/step
Model: "vgg16"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 128, 128, 3)]     0         
                                                                 
 block1_conv1 (Conv2D)       (None, 128, 128, 64)      1792      
                                                                 
 block1_conv2 (Conv2D)       (None, 128, 128, 64)      36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, 64, 64, 64)        0         
                                                                 
 block2_conv1 (Conv2D)       (None, 64, 64, 128)       73856     
                                                                 
 block2_conv2 (Conv2D)       (None, 64, 64, 128)       147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, 32, 32, 128)       0         
                                                                 
 block3_conv1 (Conv2D)       (None, 32, 32, 256)       295168    
                                                                 
 block3_conv2 (Conv2D)       (None, 32, 32, 256)       590080    
                                                                 
 block3_conv3 (Conv2D)       (None, 32, 32, 256)       590080    
                                                                 
 block3_pool (MaxPooling2D)  (None, 16, 16, 256)       0         
                                                                 
 block4_conv1 (Conv2D)       (None, 16, 16, 512)       1180160   
                                                                 
 block4_conv2 (Conv2D)       (None, 16, 16, 512)       2359808   
                                                                 
 block4_conv3 (Conv2D)       (None, 16, 16, 512)       2359808   
                                                                 
 block4_pool (MaxPooling2D)  (None, 8, 8, 512)         0         
                                                                 
 block5_conv1 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block5_conv2 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block5_conv3 (Conv2D)       (None, 8, 8, 512)         2359808   
                                                                 
 block5_pool (MaxPooling2D)  (None, 4, 4, 512)         0         
                                                                 
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
In [ ]:
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model1.layers:
    layer.trainable = False
In [ ]:
model6 = Sequential()

# Adding the convolutional part of the VGG16 model from above
model6.add(vgg_model1)

# Flattening the output of the VGG16 model because it is from a convolutional layer
model6.add(Flatten())

# Adding a fully connected dense layer
model6.add(Dense(16, activation='relu'))
model6.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model6.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt = Adam()
# Compile model
model6.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model6.summary()
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 vgg16 (Functional)          (None, 4, 4, 512)         14714688  
                                                                 
 flatten_1 (Flatten)         (None, 8192)              0         
                                                                 
 dense_2 (Dense)             (None, 16)                131088    
                                                                 
 dropout_1 (Dropout)         (None, 16)                0         
                                                                 
 dense_3 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 14845980 (56.63 MB)
Trainable params: 131292 (512.86 KB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
In [ ]:
# Epochs
epochs = 30
# Batch size
batch_size = 128

history6 = model6.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       seed=42,
                                       shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),
                    verbose=1)
Epoch 1/30
30/30 [==============================] - 31s 825ms/step - loss: 2.4394 - accuracy: 0.1433 - val_loss: 2.3099 - val_accuracy: 0.1682
Epoch 2/30
30/30 [==============================] - 20s 658ms/step - loss: 2.3494 - accuracy: 0.1662 - val_loss: 2.2923 - val_accuracy: 0.2173
Epoch 3/30
30/30 [==============================] - 19s 643ms/step - loss: 2.2901 - accuracy: 0.1845 - val_loss: 2.2151 - val_accuracy: 0.2687
Epoch 4/30
30/30 [==============================] - 19s 641ms/step - loss: 2.2438 - accuracy: 0.2208 - val_loss: 2.1504 - val_accuracy: 0.2874
Epoch 5/30
30/30 [==============================] - 20s 646ms/step - loss: 2.2162 - accuracy: 0.2264 - val_loss: 2.0888 - val_accuracy: 0.2874
Epoch 6/30
30/30 [==============================] - 20s 651ms/step - loss: 2.1796 - accuracy: 0.2194 - val_loss: 2.0147 - val_accuracy: 0.2780
Epoch 7/30
30/30 [==============================] - 19s 636ms/step - loss: 2.1476 - accuracy: 0.2393 - val_loss: 1.9719 - val_accuracy: 0.2944
Epoch 8/30
30/30 [==============================] - 20s 661ms/step - loss: 2.1171 - accuracy: 0.2501 - val_loss: 1.9975 - val_accuracy: 0.3481
Epoch 9/30
30/30 [==============================] - 19s 648ms/step - loss: 2.1047 - accuracy: 0.2597 - val_loss: 1.9454 - val_accuracy: 0.3248
Epoch 10/30
30/30 [==============================] - 19s 640ms/step - loss: 2.0974 - accuracy: 0.2544 - val_loss: 1.9068 - val_accuracy: 0.3435
Epoch 11/30
30/30 [==============================] - 20s 657ms/step - loss: 2.0694 - accuracy: 0.2581 - val_loss: 1.9249 - val_accuracy: 0.3668
Epoch 12/30
30/30 [==============================] - 25s 735ms/step - loss: 2.0615 - accuracy: 0.2670 - val_loss: 1.9023 - val_accuracy: 0.3738
Epoch 13/30
30/30 [==============================] - 19s 648ms/step - loss: 2.0362 - accuracy: 0.2681 - val_loss: 1.8797 - val_accuracy: 0.3575
Epoch 14/30
30/30 [==============================] - 25s 823ms/step - loss: 2.0460 - accuracy: 0.2716 - val_loss: 1.8429 - val_accuracy: 0.3762
Epoch 15/30
30/30 [==============================] - 35s 1s/step - loss: 2.0234 - accuracy: 0.2695 - val_loss: 1.8411 - val_accuracy: 0.3925
Epoch 16/30
30/30 [==============================] - 30s 983ms/step - loss: 2.0107 - accuracy: 0.2740 - val_loss: 1.8043 - val_accuracy: 0.3902
Epoch 17/30
30/30 [==============================] - 23s 728ms/step - loss: 2.0089 - accuracy: 0.2713 - val_loss: 1.8451 - val_accuracy: 0.3832
Epoch 18/30
30/30 [==============================] - 21s 702ms/step - loss: 1.9935 - accuracy: 0.2831 - val_loss: 1.8290 - val_accuracy: 0.3879
Epoch 19/30
30/30 [==============================] - 20s 647ms/step - loss: 1.9770 - accuracy: 0.2710 - val_loss: 1.8091 - val_accuracy: 0.4136
Epoch 20/30
30/30 [==============================] - 20s 651ms/step - loss: 1.9592 - accuracy: 0.2753 - val_loss: 1.7678 - val_accuracy: 0.4206
Epoch 21/30
30/30 [==============================] - 20s 664ms/step - loss: 1.9694 - accuracy: 0.2853 - val_loss: 1.7406 - val_accuracy: 0.4089
Epoch 22/30
30/30 [==============================] - 23s 788ms/step - loss: 1.9622 - accuracy: 0.2791 - val_loss: 1.7844 - val_accuracy: 0.4393
Epoch 23/30
30/30 [==============================] - 20s 668ms/step - loss: 1.9702 - accuracy: 0.2829 - val_loss: 1.7191 - val_accuracy: 0.4276
Epoch 24/30
30/30 [==============================] - 19s 665ms/step - loss: 1.9543 - accuracy: 0.2952 - val_loss: 1.7531 - val_accuracy: 0.4089
Epoch 25/30
30/30 [==============================] - 19s 647ms/step - loss: 1.9340 - accuracy: 0.2861 - val_loss: 1.7221 - val_accuracy: 0.4322
Epoch 26/30
30/30 [==============================] - 19s 623ms/step - loss: 1.9236 - accuracy: 0.2947 - val_loss: 1.7315 - val_accuracy: 0.4322
Epoch 27/30
30/30 [==============================] - 21s 673ms/step - loss: 1.9320 - accuracy: 0.2899 - val_loss: 1.7336 - val_accuracy: 0.4346
Epoch 28/30
30/30 [==============================] - 20s 655ms/step - loss: 1.9459 - accuracy: 0.2796 - val_loss: 1.7181 - val_accuracy: 0.4416
Epoch 29/30
30/30 [==============================] - 19s 629ms/step - loss: 1.9407 - accuracy: 0.2861 - val_loss: 1.7504 - val_accuracy: 0.4276
Epoch 30/30
30/30 [==============================] - 19s 633ms/step - loss: 1.9317 - accuracy: 0.2845 - val_loss: 1.6842 - val_accuracy: 0.4346
In [ ]:
plt.plot(history6.history['accuracy'])
plt.plot(history6.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model6.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 1s - loss: 1.6832 - accuracy: 0.4358 - 705ms/epoch - 47ms/step
In [ ]:
y_pred=model6.predict(X_test_normalized)
15/15 [==============================] - 1s 42ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 6¶

  • I introduced transfer learning using the VGG16 model using all the available layers and fitting the model using the data augmented images.
  • Although it increased the total parameters, the model is still performing poorly and false positives continue to be a problem

Model 7¶

In [ ]:
vgg_model2 = Model(inputs=vgg_model1.inputs,outputs=vgg_model1.layers[6].output)
vgg_model2.summary()
Model: "model_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 128, 128, 3)]     0         
                                                                 
 block1_conv1 (Conv2D)       (None, 128, 128, 64)      1792      
                                                                 
 block1_conv2 (Conv2D)       (None, 128, 128, 64)      36928     
                                                                 
 block1_pool (MaxPooling2D)  (None, 64, 64, 64)        0         
                                                                 
 block2_conv1 (Conv2D)       (None, 64, 64, 128)       73856     
                                                                 
 block2_conv2 (Conv2D)       (None, 64, 64, 128)       147584    
                                                                 
 block2_pool (MaxPooling2D)  (None, 32, 32, 128)       0         
                                                                 
=================================================================
Total params: 260160 (1016.25 KB)
Trainable params: 0 (0.00 Byte)
Non-trainable params: 260160 (1016.25 KB)
_________________________________________________________________
In [ ]:
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model2.layers:
    layer.trainable = False
In [ ]:
model7 = Sequential()

# Adding the convolutional part of the VGG16 model from above
model7.add(vgg_model2)

# Flattening the output of the VGG16 model because it is from a convolutional layer
model7.add(Flatten())
# Adding a fully connected dense layer
model7.add(Dense(16, activation='relu'))
model7.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model7.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt = Adam()
# Compile model
model7.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model7.summary()
Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 model_3 (Functional)        (None, 32, 32, 128)       260160    
                                                                 
 flatten_4 (Flatten)         (None, 131072)            0         
                                                                 
 dense_8 (Dense)             (None, 16)                2097168   
                                                                 
 dropout_4 (Dropout)         (None, 16)                0         
                                                                 
 dense_9 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 2357532 (8.99 MB)
Trainable params: 2097372 (8.00 MB)
Non-trainable params: 260160 (1016.25 KB)
_________________________________________________________________
In [ ]:
# Epochs
epochs = 30
# Batch size
batch_size = 128

history7 = model7.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       seed=42,
                                       shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),
                    verbose=1)
Epoch 1/30
30/30 [==============================] - 24s 782ms/step - loss: 27.2218 - accuracy: 0.1199 - val_loss: 2.4812 - val_accuracy: 0.1285
Epoch 2/30
30/30 [==============================] - 21s 702ms/step - loss: 2.4789 - accuracy: 0.1288 - val_loss: 2.4767 - val_accuracy: 0.1285
Epoch 3/30
30/30 [==============================] - 21s 676ms/step - loss: 2.4744 - accuracy: 0.1293 - val_loss: 2.4720 - val_accuracy: 0.1285
Epoch 4/30
30/30 [==============================] - 18s 615ms/step - loss: 2.4696 - accuracy: 0.1288 - val_loss: 2.4674 - val_accuracy: 0.1285
Epoch 5/30
30/30 [==============================] - 19s 625ms/step - loss: 2.4652 - accuracy: 0.1284 - val_loss: 2.4631 - val_accuracy: 0.1285
Epoch 6/30
30/30 [==============================] - 22s 733ms/step - loss: 2.4612 - accuracy: 0.1277 - val_loss: 2.4593 - val_accuracy: 0.1285
Epoch 7/30
30/30 [==============================] - 19s 609ms/step - loss: 2.4574 - accuracy: 0.1272 - val_loss: 2.4555 - val_accuracy: 0.1285
Epoch 8/30
30/30 [==============================] - 19s 637ms/step - loss: 2.4534 - accuracy: 0.1285 - val_loss: 2.4520 - val_accuracy: 0.1285
Epoch 9/30
30/30 [==============================] - 18s 609ms/step - loss: 2.4497 - accuracy: 0.1288 - val_loss: 2.4486 - val_accuracy: 0.1285
Epoch 10/30
30/30 [==============================] - 18s 609ms/step - loss: 2.4470 - accuracy: 0.1272 - val_loss: 2.4458 - val_accuracy: 0.1285
Epoch 11/30
30/30 [==============================] - 18s 601ms/step - loss: 2.4440 - accuracy: 0.1285 - val_loss: 2.4431 - val_accuracy: 0.1285
Epoch 12/30
30/30 [==============================] - 18s 593ms/step - loss: 2.4416 - accuracy: 0.1272 - val_loss: 2.4406 - val_accuracy: 0.1285
Epoch 13/30
30/30 [==============================] - 19s 621ms/step - loss: 2.4384 - accuracy: 0.1288 - val_loss: 2.4384 - val_accuracy: 0.1285
Epoch 14/30
30/30 [==============================] - 19s 649ms/step - loss: 2.4365 - accuracy: 0.1293 - val_loss: 2.4364 - val_accuracy: 0.1285
Epoch 15/30
30/30 [==============================] - 18s 632ms/step - loss: 2.4351 - accuracy: 0.1261 - val_loss: 2.4345 - val_accuracy: 0.1285
Epoch 16/30
30/30 [==============================] - 19s 613ms/step - loss: 2.4317 - accuracy: 0.1288 - val_loss: 2.4328 - val_accuracy: 0.1285
Epoch 17/30
30/30 [==============================] - 18s 608ms/step - loss: 2.4315 - accuracy: 0.1285 - val_loss: 2.4312 - val_accuracy: 0.1285
Epoch 18/30
30/30 [==============================] - 18s 604ms/step - loss: 2.4301 - accuracy: 0.1261 - val_loss: 2.4299 - val_accuracy: 0.1285
Epoch 19/30
30/30 [==============================] - 18s 614ms/step - loss: 2.4270 - accuracy: 0.1301 - val_loss: 2.4285 - val_accuracy: 0.1285
Epoch 20/30
30/30 [==============================] - 18s 603ms/step - loss: 2.4257 - accuracy: 0.1301 - val_loss: 2.4273 - val_accuracy: 0.1285
Epoch 21/30
30/30 [==============================] - 18s 595ms/step - loss: 2.4245 - accuracy: 0.1301 - val_loss: 2.4262 - val_accuracy: 0.1285
Epoch 22/30
30/30 [==============================] - 18s 612ms/step - loss: 2.4244 - accuracy: 0.1293 - val_loss: 2.4254 - val_accuracy: 0.1285
Epoch 23/30
30/30 [==============================] - 19s 612ms/step - loss: 2.4252 - accuracy: 0.1293 - val_loss: 2.4247 - val_accuracy: 0.1285
Epoch 24/30
30/30 [==============================] - 19s 623ms/step - loss: 2.4217 - accuracy: 0.1288 - val_loss: 2.4239 - val_accuracy: 0.1379
Epoch 25/30
30/30 [==============================] - 18s 605ms/step - loss: 2.4226 - accuracy: 0.1390 - val_loss: 2.4232 - val_accuracy: 0.1379
Epoch 26/30
30/30 [==============================] - 22s 732ms/step - loss: 2.4228 - accuracy: 0.1385 - val_loss: 2.4226 - val_accuracy: 0.1379
Epoch 27/30
30/30 [==============================] - 18s 635ms/step - loss: 2.4205 - accuracy: 0.1390 - val_loss: 2.4221 - val_accuracy: 0.1379
Epoch 28/30
30/30 [==============================] - 18s 609ms/step - loss: 2.4213 - accuracy: 0.1374 - val_loss: 2.4216 - val_accuracy: 0.1379
Epoch 29/30
30/30 [==============================] - 19s 620ms/step - loss: 2.4208 - accuracy: 0.1374 - val_loss: 2.4212 - val_accuracy: 0.1379
Epoch 30/30
30/30 [==============================] - 21s 700ms/step - loss: 2.4210 - accuracy: 0.1361 - val_loss: 2.4208 - val_accuracy: 0.1379
In [ ]:
plt.plot(history7.history['accuracy'])
plt.plot(history7.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model7.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 2.4194 - accuracy: 0.1368 - 326ms/epoch - 22ms/step
In [ ]:
y_pred=model7.predict(X_test_normalized)
15/15 [==============================] - 0s 19ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 7¶

  • Continuing with the use of the VGG16 architecture but cut it off after 6 layers.
  • This model performed the worst at 13% test accuracy

Model 8¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model8 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model8.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
model8.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
# Adding max pooling to reduce the size of output of first conv layer
model8.add(MaxPooling2D((2, 2), padding = 'same'))
model8.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model8.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model8.add(MaxPooling2D((2, 2), padding = 'same'))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model8.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model8.add(Dense(16, activation='relu'))
model8.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model8.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt=Adam()
# Compile model
model8.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model8.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 32)      18464     
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 32)        0         
 D)                                                              
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 563260 (2.15 MB)
Trainable params: 563260 (2.15 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
In [ ]:
history8 = model8.fit(
            X_train_normalized, y_train_encoded,
            epochs=30,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=32,
            verbose=2
)
Epoch 1/30
121/121 - 6s - loss: 2.4479 - accuracy: 0.1131 - val_loss: 2.4349 - val_accuracy: 0.1168 - 6s/epoch - 53ms/step
Epoch 2/30
121/121 - 4s - loss: 2.2894 - accuracy: 0.2371 - val_loss: 2.1182 - val_accuracy: 0.3201 - 4s/epoch - 34ms/step
Epoch 3/30
121/121 - 4s - loss: 2.1361 - accuracy: 0.2802 - val_loss: 2.0835 - val_accuracy: 0.3481 - 4s/epoch - 35ms/step
Epoch 4/30
121/121 - 4s - loss: 2.0529 - accuracy: 0.2969 - val_loss: 1.9439 - val_accuracy: 0.3575 - 4s/epoch - 34ms/step
Epoch 5/30
121/121 - 4s - loss: 2.0064 - accuracy: 0.3112 - val_loss: 2.0102 - val_accuracy: 0.3248 - 4s/epoch - 34ms/step
Epoch 6/30
121/121 - 4s - loss: 1.9876 - accuracy: 0.3065 - val_loss: 1.8199 - val_accuracy: 0.3808 - 4s/epoch - 33ms/step
Epoch 7/30
121/121 - 4s - loss: 1.9569 - accuracy: 0.3236 - val_loss: 1.7626 - val_accuracy: 0.4159 - 4s/epoch - 32ms/step
Epoch 8/30
121/121 - 4s - loss: 1.9218 - accuracy: 0.3314 - val_loss: 1.8143 - val_accuracy: 0.3621 - 4s/epoch - 34ms/step
Epoch 9/30
121/121 - 4s - loss: 1.8911 - accuracy: 0.3343 - val_loss: 1.7515 - val_accuracy: 0.4136 - 4s/epoch - 35ms/step
Epoch 10/30
121/121 - 4s - loss: 1.8526 - accuracy: 0.3488 - val_loss: 1.7654 - val_accuracy: 0.3692 - 4s/epoch - 33ms/step
Epoch 11/30
121/121 - 4s - loss: 1.8520 - accuracy: 0.3322 - val_loss: 1.6924 - val_accuracy: 0.3808 - 4s/epoch - 32ms/step
Epoch 12/30
121/121 - 4s - loss: 1.8454 - accuracy: 0.3353 - val_loss: 1.6804 - val_accuracy: 0.4159 - 4s/epoch - 33ms/step
Epoch 13/30
121/121 - 4s - loss: 1.8221 - accuracy: 0.3426 - val_loss: 1.7031 - val_accuracy: 0.3808 - 4s/epoch - 34ms/step
Epoch 14/30
121/121 - 4s - loss: 1.7809 - accuracy: 0.3533 - val_loss: 1.7013 - val_accuracy: 0.3995 - 4s/epoch - 32ms/step
Epoch 15/30
121/121 - 4s - loss: 1.7747 - accuracy: 0.3517 - val_loss: 1.6662 - val_accuracy: 0.4159 - 4s/epoch - 32ms/step
Epoch 16/30
121/121 - 4s - loss: 1.7691 - accuracy: 0.3494 - val_loss: 1.6319 - val_accuracy: 0.4159 - 4s/epoch - 33ms/step
Epoch 17/30
121/121 - 4s - loss: 1.7410 - accuracy: 0.3613 - val_loss: 1.6213 - val_accuracy: 0.4136 - 4s/epoch - 34ms/step
Epoch 18/30
121/121 - 4s - loss: 1.7357 - accuracy: 0.3665 - val_loss: 1.6667 - val_accuracy: 0.4136 - 4s/epoch - 33ms/step
Epoch 19/30
121/121 - 4s - loss: 1.7210 - accuracy: 0.3637 - val_loss: 1.6320 - val_accuracy: 0.4182 - 4s/epoch - 32ms/step
Epoch 20/30
121/121 - 4s - loss: 1.7157 - accuracy: 0.3696 - val_loss: 1.6198 - val_accuracy: 0.4229 - 4s/epoch - 33ms/step
Epoch 21/30
121/121 - 4s - loss: 1.7162 - accuracy: 0.3689 - val_loss: 1.6287 - val_accuracy: 0.4299 - 4s/epoch - 35ms/step
Epoch 22/30
121/121 - 4s - loss: 1.7219 - accuracy: 0.3600 - val_loss: 1.6148 - val_accuracy: 0.4136 - 4s/epoch - 33ms/step
Epoch 23/30
121/121 - 4s - loss: 1.6884 - accuracy: 0.3754 - val_loss: 1.6036 - val_accuracy: 0.4393 - 4s/epoch - 32ms/step
Epoch 24/30
121/121 - 4s - loss: 1.6582 - accuracy: 0.3787 - val_loss: 1.6366 - val_accuracy: 0.4322 - 4s/epoch - 33ms/step
Epoch 25/30
121/121 - 4s - loss: 1.6642 - accuracy: 0.3733 - val_loss: 1.6247 - val_accuracy: 0.4369 - 4s/epoch - 34ms/step
Epoch 26/30
121/121 - 4s - loss: 1.6388 - accuracy: 0.3832 - val_loss: 1.6590 - val_accuracy: 0.4299 - 4s/epoch - 33ms/step
Epoch 27/30
121/121 - 4s - loss: 1.6508 - accuracy: 0.3754 - val_loss: 1.6364 - val_accuracy: 0.4369 - 4s/epoch - 32ms/step
Epoch 28/30
121/121 - 4s - loss: 1.6488 - accuracy: 0.3878 - val_loss: 1.6369 - val_accuracy: 0.4299 - 4s/epoch - 33ms/step
Epoch 29/30
121/121 - 4s - loss: 1.6425 - accuracy: 0.3881 - val_loss: 1.6199 - val_accuracy: 0.4416 - 4s/epoch - 33ms/step
Epoch 30/30
121/121 - 4s - loss: 1.6291 - accuracy: 0.3949 - val_loss: 1.7153 - val_accuracy: 0.4439 - 4s/epoch - 33ms/step
In [ ]:
plt.plot(history8.history['accuracy'])
plt.plot(history8.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model8.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 1.8101 - accuracy: 0.4232 - 155ms/epoch - 10ms/step
In [ ]:
y_pred=model8.predict(X_test_normalized)
15/15 [==============================] - 0s 9ms/step
In [ ]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()

Observations for Model 8¶

  • I removed the batch normalization optimizer and but the model is still performing sub 50% accuracy

Model 9¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model9 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model9.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
model9.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
# Adding max pooling to reduce the size of output of first conv layer
model9.add(MaxPooling2D((2, 2), padding = 'same'))
model9.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model9.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model9.add(MaxPooling2D((2, 2), padding = 'same'))
model9.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model9.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model9.add(Dense(16, activation='relu'))
model9.add(Dropout(0.3))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model9.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt=Adam()
# Compile model
model9.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model9.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 128, 128, 64)      1792      
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 32)      18464     
                                                                 
 max_pooling2d (MaxPooling2  (None, 64, 64, 32)        0         
 D)                                                              
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 max_pooling2d_1 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization (Batch  (None, 32, 32, 32)        128       
 Normalization)                                                  
                                                                 
 flatten (Flatten)           (None, 32768)             0         
                                                                 
 dense (Dense)               (None, 16)                524304    
                                                                 
 dropout (Dropout)           (None, 16)                0         
                                                                 
 dense_1 (Dense)             (None, 12)                204       
                                                                 
=================================================================
Total params: 563388 (2.15 MB)
Trainable params: 563324 (2.15 MB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [ ]:
history9 = model9.fit(
            X_train_normalized, y_train_encoded,
            epochs=30,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=32,
            verbose=2)
Epoch 1/30
121/121 - 7s - loss: 2.2389 - accuracy: 0.2345 - val_loss: 2.4250 - val_accuracy: 0.1379 - 7s/epoch - 56ms/step
Epoch 2/30
121/121 - 5s - loss: 1.8594 - accuracy: 0.3540 - val_loss: 2.3103 - val_accuracy: 0.2290 - 5s/epoch - 38ms/step
Epoch 3/30
121/121 - 4s - loss: 1.5119 - accuracy: 0.4669 - val_loss: 2.0657 - val_accuracy: 0.2266 - 4s/epoch - 35ms/step
Epoch 4/30
121/121 - 5s - loss: 1.3546 - accuracy: 0.5233 - val_loss: 2.7100 - val_accuracy: 0.2897 - 5s/epoch - 39ms/step
Epoch 5/30
121/121 - 5s - loss: 1.2641 - accuracy: 0.5402 - val_loss: 2.1385 - val_accuracy: 0.3855 - 5s/epoch - 42ms/step
Epoch 6/30
121/121 - 5s - loss: 1.1213 - accuracy: 0.5984 - val_loss: 2.9967 - val_accuracy: 0.2056 - 5s/epoch - 41ms/step
Epoch 7/30
121/121 - 4s - loss: 1.0643 - accuracy: 0.6119 - val_loss: 1.0884 - val_accuracy: 0.6379 - 4s/epoch - 35ms/step
Epoch 8/30
121/121 - 4s - loss: 0.9526 - accuracy: 0.6532 - val_loss: 2.9286 - val_accuracy: 0.4953 - 4s/epoch - 33ms/step
Epoch 9/30
121/121 - 4s - loss: 0.8522 - accuracy: 0.6914 - val_loss: 1.2920 - val_accuracy: 0.5935 - 4s/epoch - 34ms/step
Epoch 10/30
121/121 - 4s - loss: 0.8084 - accuracy: 0.6982 - val_loss: 1.1995 - val_accuracy: 0.6215 - 4s/epoch - 34ms/step
Epoch 11/30
121/121 - 4s - loss: 0.7568 - accuracy: 0.7193 - val_loss: 1.1355 - val_accuracy: 0.6589 - 4s/epoch - 33ms/step
Epoch 12/30
121/121 - 4s - loss: 0.7497 - accuracy: 0.7219 - val_loss: 1.0236 - val_accuracy: 0.6682 - 4s/epoch - 32ms/step
Epoch 13/30
121/121 - 4s - loss: 0.6690 - accuracy: 0.7463 - val_loss: 2.0394 - val_accuracy: 0.5397 - 4s/epoch - 34ms/step
Epoch 14/30
121/121 - 4s - loss: 0.6275 - accuracy: 0.7609 - val_loss: 1.2165 - val_accuracy: 0.6589 - 4s/epoch - 34ms/step
Epoch 15/30
121/121 - 4s - loss: 0.6156 - accuracy: 0.7697 - val_loss: 3.2296 - val_accuracy: 0.3411 - 4s/epoch - 32ms/step
Epoch 16/30
121/121 - 4s - loss: 0.6210 - accuracy: 0.7655 - val_loss: 1.0700 - val_accuracy: 0.6869 - 4s/epoch - 32ms/step
Epoch 17/30
121/121 - 4s - loss: 0.5703 - accuracy: 0.7767 - val_loss: 1.1539 - val_accuracy: 0.6916 - 4s/epoch - 37ms/step
Epoch 18/30
121/121 - 4s - loss: 0.5327 - accuracy: 0.7931 - val_loss: 1.6504 - val_accuracy: 0.6519 - 4s/epoch - 34ms/step
Epoch 19/30
121/121 - 4s - loss: 0.5198 - accuracy: 0.8004 - val_loss: 2.2241 - val_accuracy: 0.5888 - 4s/epoch - 34ms/step
Epoch 20/30
121/121 - 4s - loss: 0.5267 - accuracy: 0.7970 - val_loss: 1.3839 - val_accuracy: 0.6729 - 4s/epoch - 34ms/step
Epoch 21/30
121/121 - 4s - loss: 0.5000 - accuracy: 0.8037 - val_loss: 2.1263 - val_accuracy: 0.5140 - 4s/epoch - 35ms/step
Epoch 22/30
121/121 - 4s - loss: 0.5177 - accuracy: 0.8017 - val_loss: 1.7792 - val_accuracy: 0.6285 - 4s/epoch - 34ms/step
Epoch 23/30
121/121 - 4s - loss: 0.4911 - accuracy: 0.8079 - val_loss: 3.0073 - val_accuracy: 0.4626 - 4s/epoch - 33ms/step
Epoch 24/30
121/121 - 4s - loss: 0.4782 - accuracy: 0.8175 - val_loss: 1.8606 - val_accuracy: 0.6215 - 4s/epoch - 33ms/step
Epoch 25/30
121/121 - 4s - loss: 0.4756 - accuracy: 0.8149 - val_loss: 1.6411 - val_accuracy: 0.6542 - 4s/epoch - 36ms/step
Epoch 26/30
121/121 - 4s - loss: 0.4248 - accuracy: 0.8303 - val_loss: 1.7797 - val_accuracy: 0.6355 - 4s/epoch - 34ms/step
Epoch 27/30
121/121 - 4s - loss: 0.4658 - accuracy: 0.8274 - val_loss: 1.8125 - val_accuracy: 0.6542 - 4s/epoch - 35ms/step
Epoch 28/30
121/121 - 4s - loss: 0.4123 - accuracy: 0.8446 - val_loss: 1.7110 - val_accuracy: 0.6355 - 4s/epoch - 35ms/step
Epoch 29/30
121/121 - 4s - loss: 0.4133 - accuracy: 0.8425 - val_loss: 2.0162 - val_accuracy: 0.6332 - 4s/epoch - 35ms/step
Epoch 30/30
121/121 - 4s - loss: 0.4089 - accuracy: 0.8420 - val_loss: 1.4466 - val_accuracy: 0.6659 - 4s/epoch - 34ms/step
In [ ]:
plt.plot(history9.history['accuracy'])
plt.plot(history9.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model9.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 1.5982 - accuracy: 0.6168 - 257ms/epoch - 17ms/step
In [ ]:
y_pred_prob = model9.predict(X_test_normalized)
15/15 [==============================] - 0s 14ms/step
In [ ]:
# Determine the threshold based on ROC AUC
thresholds = []
for i in range(12):  # Change 12 to the actual number of classes
    roc_auc = roc_auc_score(y_test_encoded[:, i], y_pred_prob[:, i])
    fpr, tpr, threshold = roc_curve(y_test_encoded[:, i], y_pred_prob[:, i])
    best_threshold_index = np.argmax(tpr - fpr)
    best_threshold = threshold[best_threshold_index]
    thresholds.append(best_threshold)

# Use the determined thresholds for prediction
y_pred_binary = np.zeros_like(y_pred_prob)
for i in range(12):  # Change 12 to the actual number of classes
    y_pred_binary[:, i] = (y_pred_prob[:, i] > thresholds[i]).astype(int)

# Obtain the accuracy using the new predicted binary values
accuracy_roc_auc = accuracy_score(y_test_encoded, y_pred_binary)
print(f"Test Accuracy (ROC AUC Threshold): {accuracy_roc_auc}")
Test Accuracy (ROC AUC Threshold): 0.3136842105263158
In [ ]:
# Plotting the Confusion Matrix using confusion matrix() function
conf_matrix = confusion_matrix(
    y_test_encoded.argmax(axis=1),
    y_pred_binary.argmax(axis=1)
)

# Display the confusion matrix using seaborn
plt.figure(figsize=(10, 8))
sns.heatmap(conf_matrix, annot=True, fmt="d", xticklabels=np.arange(12), yticklabels=np.arange(12))
plt.title('Confusion Matrix')
plt.xlabel('Predicted')
plt.ylabel('True')
plt.show()

Observations for Model 7¶

  • Introduced the ROC AUC to determine the optimal threshold for our prediction which resulted in improvement of the accuracy scores to 85% training accuracy, 67% validation accuracy, and 62% testing accuracy

Model 10¶

In [ ]:
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [ ]:
# Intializing a sequential model
model10 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model10.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(128, 128, 3)))
model10.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
# Adding max pooling to reduce the size of output of first conv layer
model10.add(MaxPooling2D((2, 2), padding = 'same'))
model10.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model10.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model10.add(MaxPooling2D((2, 2), padding = 'same'))
model10.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model10.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model10.add(Dense(32, activation='relu'))
model10.add(Dropout(0.5))
# Adding the output layer with 10 neurons and activation functions as softmax since this is a multi-class classification problem
model10.add(Dense(12, activation='softmax'))

#using Adam optimizer
opt=Adam()
# Compile model
model10.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model10.summary()
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d_4 (Conv2D)           (None, 128, 128, 64)      1792      
                                                                 
 conv2d_5 (Conv2D)           (None, 128, 128, 32)      18464     
                                                                 
 max_pooling2d_2 (MaxPoolin  (None, 64, 64, 32)        0         
 g2D)                                                            
                                                                 
 conv2d_6 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 conv2d_7 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 max_pooling2d_3 (MaxPoolin  (None, 32, 32, 32)        0         
 g2D)                                                            
                                                                 
 batch_normalization_1 (Bat  (None, 32, 32, 32)        128       
 chNormalization)                                                
                                                                 
 flatten_1 (Flatten)         (None, 32768)             0         
                                                                 
 dense_2 (Dense)             (None, 32)                1048608   
                                                                 
 dropout_1 (Dropout)         (None, 32)                0         
                                                                 
 dense_3 (Dense)             (None, 12)                396       
                                                                 
=================================================================
Total params: 1087884 (4.15 MB)
Trainable params: 1087820 (4.15 MB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [ ]:
# Code to monitor val_accuracy
reduce_lr = ReduceLROnPlateau(monitor='val_accuracy',
                                            patience=3,
                                            verbose=1,
                                            factor=0.5,
                                            min_lr=0.00001)

history10 = model10.fit(
    X_train_normalized, y_train_encoded,
    epochs=30,
    validation_data=(X_val_normalized, y_val_encoded),
    batch_size=32,
    verbose=2,
    callbacks=[reduce_lr]
)
Epoch 1/30
121/121 - 7s - loss: 2.1359 - accuracy: 0.2789 - val_loss: 2.4063 - val_accuracy: 0.1822 - lr: 0.0010 - 7s/epoch - 54ms/step
Epoch 2/30
121/121 - 4s - loss: 1.7066 - accuracy: 0.4167 - val_loss: 2.3228 - val_accuracy: 0.1729 - lr: 0.0010 - 4s/epoch - 35ms/step
Epoch 3/30
121/121 - 4s - loss: 1.4879 - accuracy: 0.4926 - val_loss: 1.9852 - val_accuracy: 0.4252 - lr: 0.0010 - 4s/epoch - 34ms/step
Epoch 4/30
121/121 - 4s - loss: 1.4029 - accuracy: 0.5147 - val_loss: 1.6024 - val_accuracy: 0.5117 - lr: 0.0010 - 4s/epoch - 34ms/step
Epoch 5/30
121/121 - 4s - loss: 1.2788 - accuracy: 0.5656 - val_loss: 1.5945 - val_accuracy: 0.4907 - lr: 0.0010 - 4s/epoch - 35ms/step
Epoch 6/30
121/121 - 4s - loss: 1.1737 - accuracy: 0.5841 - val_loss: 1.1178 - val_accuracy: 0.6402 - lr: 0.0010 - 4s/epoch - 35ms/step
Epoch 7/30
121/121 - 4s - loss: 1.0978 - accuracy: 0.6059 - val_loss: 1.2592 - val_accuracy: 0.6051 - lr: 0.0010 - 4s/epoch - 33ms/step
Epoch 8/30
121/121 - 4s - loss: 1.0196 - accuracy: 0.6278 - val_loss: 1.2044 - val_accuracy: 0.6425 - lr: 0.0010 - 4s/epoch - 33ms/step
Epoch 9/30
121/121 - 4s - loss: 0.9620 - accuracy: 0.6592 - val_loss: 2.2442 - val_accuracy: 0.4556 - lr: 0.0010 - 4s/epoch - 34ms/step
Epoch 10/30
121/121 - 4s - loss: 0.9417 - accuracy: 0.6543 - val_loss: 1.0824 - val_accuracy: 0.6659 - lr: 0.0010 - 4s/epoch - 34ms/step
Epoch 11/30
121/121 - 4s - loss: 0.8503 - accuracy: 0.6985 - val_loss: 1.2646 - val_accuracy: 0.6145 - lr: 0.0010 - 4s/epoch - 34ms/step
Epoch 12/30
121/121 - 4s - loss: 0.8313 - accuracy: 0.6959 - val_loss: 1.9335 - val_accuracy: 0.5654 - lr: 0.0010 - 4s/epoch - 33ms/step
Epoch 13/30

Epoch 13: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
121/121 - 4s - loss: 0.7808 - accuracy: 0.7073 - val_loss: 1.6651 - val_accuracy: 0.6285 - lr: 0.0010 - 4s/epoch - 34ms/step
Epoch 14/30
121/121 - 4s - loss: 0.6590 - accuracy: 0.7528 - val_loss: 1.2815 - val_accuracy: 0.6121 - lr: 5.0000e-04 - 4s/epoch - 33ms/step
Epoch 15/30
121/121 - 4s - loss: 0.6320 - accuracy: 0.7624 - val_loss: 1.0678 - val_accuracy: 0.6986 - lr: 5.0000e-04 - 4s/epoch - 33ms/step
Epoch 16/30
121/121 - 4s - loss: 0.6340 - accuracy: 0.7598 - val_loss: 0.9532 - val_accuracy: 0.7173 - lr: 5.0000e-04 - 4s/epoch - 34ms/step
Epoch 17/30
121/121 - 4s - loss: 0.5672 - accuracy: 0.7837 - val_loss: 1.1993 - val_accuracy: 0.6449 - lr: 5.0000e-04 - 4s/epoch - 34ms/step
Epoch 18/30
121/121 - 4s - loss: 0.5615 - accuracy: 0.7814 - val_loss: 1.2611 - val_accuracy: 0.7150 - lr: 5.0000e-04 - 4s/epoch - 33ms/step
Epoch 19/30

Epoch 19: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
121/121 - 4s - loss: 0.5449 - accuracy: 0.7892 - val_loss: 1.2877 - val_accuracy: 0.6916 - lr: 5.0000e-04 - 4s/epoch - 36ms/step
Epoch 20/30
121/121 - 4s - loss: 0.5133 - accuracy: 0.7985 - val_loss: 1.0661 - val_accuracy: 0.7336 - lr: 2.5000e-04 - 4s/epoch - 35ms/step
Epoch 21/30
121/121 - 4s - loss: 0.4752 - accuracy: 0.8131 - val_loss: 1.1244 - val_accuracy: 0.7453 - lr: 2.5000e-04 - 4s/epoch - 34ms/step
Epoch 22/30
121/121 - 4s - loss: 0.4642 - accuracy: 0.8123 - val_loss: 1.1950 - val_accuracy: 0.7150 - lr: 2.5000e-04 - 4s/epoch - 34ms/step
Epoch 23/30
121/121 - 4s - loss: 0.4611 - accuracy: 0.8225 - val_loss: 1.1081 - val_accuracy: 0.7336 - lr: 2.5000e-04 - 4s/epoch - 33ms/step
Epoch 24/30

Epoch 24: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
121/121 - 4s - loss: 0.4535 - accuracy: 0.8245 - val_loss: 1.0947 - val_accuracy: 0.7196 - lr: 2.5000e-04 - 4s/epoch - 34ms/step
Epoch 25/30
121/121 - 4s - loss: 0.4328 - accuracy: 0.8349 - val_loss: 1.1109 - val_accuracy: 0.7407 - lr: 1.2500e-04 - 4s/epoch - 35ms/step
Epoch 26/30
121/121 - 4s - loss: 0.3989 - accuracy: 0.8409 - val_loss: 1.1505 - val_accuracy: 0.7523 - lr: 1.2500e-04 - 4s/epoch - 36ms/step
Epoch 27/30
121/121 - 4s - loss: 0.4314 - accuracy: 0.8264 - val_loss: 1.1505 - val_accuracy: 0.7407 - lr: 1.2500e-04 - 4s/epoch - 33ms/step
Epoch 28/30
121/121 - 4s - loss: 0.4218 - accuracy: 0.8316 - val_loss: 1.1382 - val_accuracy: 0.7430 - lr: 1.2500e-04 - 4s/epoch - 34ms/step
Epoch 29/30

Epoch 29: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
121/121 - 4s - loss: 0.3971 - accuracy: 0.8438 - val_loss: 1.1789 - val_accuracy: 0.7290 - lr: 1.2500e-04 - 4s/epoch - 35ms/step
Epoch 30/30
121/121 - 4s - loss: 0.3798 - accuracy: 0.8440 - val_loss: 1.1647 - val_accuracy: 0.7430 - lr: 6.2500e-05 - 4s/epoch - 33ms/step
In [ ]:
plt.plot(history10.history['accuracy'])
plt.plot(history10.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [ ]:
accuracy = model10.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 1.1067 - accuracy: 0.7179 - 256ms/epoch - 17ms/step
In [ ]:
y_pred_prob=model10.predict(X_test_normalized)
15/15 [==============================] - 0s 13ms/step
In [ ]:
# Determine the threshold based on ROC AUC
thresholds = []
for i in range(12):  # Change 12 to the actual number of classes
    roc_auc = roc_auc_score(y_test_encoded[:, i], y_pred_prob[:, i])
    fpr, tpr, threshold = roc_curve(y_test_encoded[:, i], y_pred_prob[:, i])
    best_threshold_index = np.argmax(tpr - fpr)
    best_threshold = threshold[best_threshold_index]
    thresholds.append(best_threshold)

# Use the determined thresholds for prediction
y_pred_binary = np.zeros_like(y_pred_prob)
for i in range(12):  # Change 12 to the actual number of classes
    y_pred_binary[:, i] = (y_pred_prob[:, i] > thresholds[i]).astype(int)

# Obtain the accuracy using the new predicted binary values
accuracy_roc_auc = accuracy_score(y_test_encoded, y_pred_binary)
print(f"Test Accuracy (ROC AUC Threshold): {accuracy_roc_auc}")
Test Accuracy (ROC AUC Threshold): 0.45473684210526316
In [ ]:
# Plotting the Confusion Matrix using confusion matrix() function
conf_matrix = confusion_matrix(
    y_test_encoded.argmax(axis=1),
    y_pred_binary.argmax(axis=1)
)

# Display the confusion matrix using seaborn
plt.figure(figsize=(10, 8))
sns.heatmap(conf_matrix, annot=True, fmt="d", xticklabels=np.arange(12), yticklabels=np.arange(12))
plt.title('Confusion Matrix')
plt.xlabel('Predicted')
plt.ylabel('True')
plt.show()

Observations for Model 10¶

  • I adjusted this model to have 32 hidden layers in the ANN, and increased the dropout rate from 0.3 to 0.5. I also reduced the learning rate and used ROC AUC to find the optimal threshold.
  • These adjustments resulted in an 84% training accuracy, 74% validation accuracy, and 72% testing accuracy which our best performing model thusfar

Final Model¶

Accuracy comparisons¶

In [ ]:
pd.DataFrame({'Models':['Model1', 'Model2', 'Model3', 'Model4', 'Model5', 'Model6', 'Model7', 'Model8', 'Model9','Model10'],
              'Train Accuracy':['43%','87%','59%','42%','51%','29%','14%','39%','84%','84%'],
              'Validation Accuracy':['49%','60%','45%','23%','47%','43%','14%','44%','67%','74%'],
              'Test Accuracy':['50%','61%','46%','24%','44%','44%','14%','42%','62%','72%']
              })
Out[ ]:
Models Train Accuracy Validation Accuracy Test Accuracy
0 Model1 43% 49% 50%
1 Model2 87% 60% 61%
2 Model3 59% 45% 46%
3 Model4 42% 23% 24%
4 Model5 51% 47% 44%
5 Model6 29% 43% 44%
6 Model7 14% 14% 14%
7 Model8 39% 44% 42%
8 Model9 84% 67% 62%
9 Model10 84% 74% 72%

Visualizing the prediction¶

In [ ]:
rows = 2
cols = 5

# Create a figure with the specified number of rows and columns
fig, axes = plt.subplots(rows, cols, figsize=(15, 10))

# Flatten the axes for easy iteration
axes = axes.flatten()

for i in range(rows * cols):
    random_index = np.random.randint(0, len(X_test_normalized))
    ax = axes[i]

    # Display the image
    ax.imshow(X_test[random_index])

    # Get the predicted label, probability, and true label
    pred_label = np.argmax(y_pred_binary[random_index])
    pred_proba = y_pred_prob[random_index, pred_label]
    true_label = np.argmax(y_test_encoded[random_index])

    # Set the title with actual, predicted, and probability information
    ax.set_title(f"Actual: {true_label}\nPredicted: {pred_label}\nProbability: {pred_proba:.3f}", fontsize=10)

    # Remove the x and y ticks for better visualization
    ax.set_xticks([])
    ax.set_yticks([])

# Adjust layout for better spacing
plt.tight_layout()
plt.show()

Actionable Insights and Business Recommendations¶

  • We can observe from Model 10 that there are still false positives primarily in the 4, 5, and 6 classes.
  • Improvements can be made by balancing the dataset to increase the available data points for the minority classes.

In [ ]: